System, Method and Apparatus for Magnetic Tracking of Ultrasound Probe and Generation of 3D Visualization Thereof

ABSTRACT

Disclosed herein is a magnetic-based tracking system for tracking an ultrasound probe to create a three-dimensional visualization. The system includes a reference device including a reference magnet; an ultrasound probe including an ultrasound acoustic transducer or acoustic array that acquires ultrasound images and a magnetometer that detects a magnetic field generated by the reference magnet. The ultrasound probe couples a first ultrasound image with a first magnetic field strength, wherein both of the first ultrasound image is received and the first magnetic field strength is detected at a first time; and a console including a processor and non-transitory computer-readable medium having stored thereon a plurality of processor executed logic modules that perform operations including receiving and recording a plurality of coupling of ultrasound images and detected magnetic field strengths, and generating the 3D visualization from the ultrasound images by aligning the ultrasound images in accordance with a corresponding detected magnetic field strength.

PRIORITY

This application claims the benefit of priority to U.S. ProvisionalApplication No. 63/054,622, filed Jul. 21, 2020, which is incorporatedby reference in its entirety into this application.

SUMMARY

Briefly summarized, embodiments disclosed herein are directed tosystems, methods, and devices for magnetic tracking of ultrasound probesand use of the magnetic tracking of ultrasound probes to generate 3Dvisualization of target vessels.

Blood vessel cannulation can be difficult in some patients. One problemthat often arises with blood vessel cannulation is difficulty invisualizing the target vessel and other details important forcannulation including vessel movement, vessel size, depth, proximity tounintended structures or even impediments within the target vesselincluding valves, stenosis, thrombosis etc.

In some embodiments, a magnetic-based tracking system for tracking anultrasound probe to create a three-dimensional (3D) visualizationincludes a reference device including a reference magnet; an ultrasoundprobe including a magnetometer configured to detect a magnetic fieldgenerated by the reference magnet, wherein the ultrasound probe isconfigured to couple a first ultrasound image with a first magneticfield strength, and wherein the first ultrasound image is received at afirst time, and the first magnetic field strength is detected at thefirst time. The system further includes a console including a processorand non-transitory computer-readable medium having stored thereon aplurality of logic modules that, when executed by the processor, areconfigured to perform operations including receiving a plurality ofcouplings of ultrasound images and detected magnetic field strengths,recording the plurality of couplings of ultrasound images and detectedmagnetic field strengths, and generating the 3D visualization from theultrasound images by aligning each of the ultrasound images inaccordance with a corresponding detected magnetic field strength.

In some embodiments, the magnetic based tracking system includes wherethe reference device is a cuff like structure that wraps around a bodysegment to be imaged.

In some embodiments, the magnetic based tracking system includes wherethe reference device is a U-shaped structure and is configured to allowthe body segment to be placed within the U-shaped structure for imaging.

Also disclosed is a magnetic based tracking system for tracking anultrasound probe to create a three-dimensional (3D) visualization,including a reference device including a magnetometer that detects amagnetic field generated by a reference magnet and creates a timestampfor a magnetic field strength reading; an ultrasound probe including anultrasound acoustic transducer or acoustic array that acquiresultrasound images and creates a timestamp for each specific ultrasoundimage, and a reference magnet that is configured to generate a magneticfield. The system also includes a console including a processor andnon-transitory computer-readable medium having stored thereon aplurality of logic modules that, when executed by the processor, areconfigured to perform operations including receiving a plurality ofultrasound images and detected magnetic field strengths, coupling aplurality of ultrasound images and detected magnetic field strengths bytheir timestamps, recording the plurality of couplings of ultrasoundimages and detected magnetic field strengths, and generating the 3Dvisualization from the ultrasound images by aligning each of theultrasound images in accordance with a corresponding detected magneticfield strength.

In some embodiments, the magnetic based tracking system includes wherethe reference device is a cuff like structure that wraps around a bodysegment to be imaged.

In some embodiments, the magnetic based tracking system includes wherethe reference device is a U-shaped structure and is configured to allowthe body segment to be placed within the U-shaped structure for imaging.

Also disclosed is a method of creating a 3D image using a magnetic basedtracking system for tracking an ultrasound probe including configuringthe reference device around the body segment to be imaged; advancingultrasound probe on skin surface of the body segment to be imaged;capturing time stamped ultrasound images while simultaneously detectingtime stamped magnetic field strength of a reference magnet bymagnetometer; determining distance between reference magnet andmagnetometer; and stitching together ultrasound images using magneticfield strength data and time stamps to create a 3D image.

In some embodiments, the method includes where configuring the referencedevice includes the reference device being a cuff like structure andincluding a reference magnet.

In some embodiments, the method includes where advancing the ultrasoundprobe includes the ultrasound probe including a magnetometer.

In some embodiments, the method includes where configuring the referencedevice includes the reference device being a U-shaped structure and isconfigured to allow the bodily appendage to be placed within theU-shaped structure for imaging.

In some embodiments, the method includes where the stitching togetherthe ultrasound images includes using only the magnetic field strengthdata.

In some embodiments, the method includes where configuring the referencedevice includes the reference device being a cuff-like structure andincluding the magnetometer.

In some embodiments, the method includes where advancing the ultrasoundprobe includes the ultrasound probe including the reference magnet.

These and other features of the concepts provided herein will becomemore apparent to those of skill in the art in view of the accompanyingdrawings and following description, which disclose particularembodiments of such concepts in greater detail.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the disclosure are illustrated by way of example and notby way of limitation in the figures of the accompanying drawings, inwhich like references indicate similar elements and in which:

FIG. 1 illustrates a block diagram depicting various elements of amagnetic-based tracking system for an ultrasound probe and other medicalcomponents to create a 3D image according to some embodiments;

FIG. 2A illustrates a top view of the magnetic-based tracking systemincluding the ultrasound probe of FIG. 1 according to some embodiments;

FIG. 2B illustrates a top view of a magnetic-based tracking systemincluding a remote-control tourniquet, a remote-control heating elementand the ultrasound probe of FIG. 1 according to some embodiments;

FIG. 3A illustrates a side view of the magnetic-based tracking systemincluding the ultrasound probe of FIG. 2A according to some embodiments;

FIG. 3B illustrates a front view of the magnetic-based tracking systemincluding the ultrasound probe and console of FIG. 2A according to someembodiments;

FIG. 4 illustrates a side view of the magnetic based tracking systemincluding the ultrasound probe of FIG. 3 according to some embodiments;

FIG. 5 is a flowchart illustrating an exemplary method for using amagnetic-based tracking system for an ultrasound probe of FIG. 1 tocreate a 3D image according to some embodiments;

FIG. 6 illustrates a block diagram depicting various elements of amagnetic-based tracking system for an ultrasound probe and other medicalcomponents to create a 3D image according to some embodiments;

FIG. 7 illustrates a top view of the magnetic-based tracking systemincluding the ultrasound probe of FIG. 6 according to some embodiments;

FIG. 8 illustrates a side view of the magnetic-based tracking systemincluding the ultrasound probe of FIG. 7 according to some embodiments;

FIG. 9 illustrates a side view of the magnetic based tracking systemincluding the ultrasound probe of FIG. 8 according to some embodiments;and

FIG. 10 is a flowchart illustrating an exemplary method for using amagnetic-based tracking system for an ultrasound probe of FIG. 6 tocreate a 3D image according to some embodiments.

DETAILED DESCRIPTION

Before some particular embodiments are disclosed in greater detail, itshould be understood that the particular embodiments disclosed herein donot limit the scope of the concepts provided herein. It should also beunderstood that a particular embodiment disclosed herein can havefeatures that can be readily separated from the particular embodimentand optionally combined with or substituted for features of any of anumber of other embodiments disclosed herein.

Regarding terms used herein, it should also be understood the terms arefor the purpose of describing some particular embodiments, and the termsdo not limit the scope of the concepts provided herein. Ordinal numbers(e.g., first, second, third, etc.) are generally used to distinguish oridentify different features or steps in a group of features or steps,and do not supply a serial or numerical limitation. For example,“first,” “second,” and “third” features or steps need not necessarilyappear in that order, and the particular embodiments including suchfeatures or steps need not necessarily be limited to the three featuresor steps. Labels such as “left,” “right,” “top,” “bottom,” “front,”“back,” and the like are used for convenience and are not intended toimply, for example, any particular fixed location, orientation, ordirection. Instead, such labels are used to reflect, for example,relative location, orientation, or directions. Singular forms of “a,”“an,” and “the” include plural references unless the context clearlydictates otherwise.

With respect to “proximal,” a “proximal portion” or a “proximal endportion” of, for example, a probe disclosed herein includes a portion ofthe probe intended to be near a clinician when the probe is used on apatient. Likewise, a “proximal length” of, for example, the probeincludes a length of the probe intended to be near the clinician whenthe probe is used on the patient. A “proximal end” of, for example, theprobe includes an end of the probe intended to be near the clinicianwhen the probe is used on the patient. The proximal portion, theproximal end portion, or the proximal length of the probe can includethe proximal end of the probe; however, the proximal portion, theproximal end portion, or the proximal length of the probe need notinclude the proximal end of the probe. That is, unless context suggestsotherwise, the proximal portion, the proximal end portion, or theproximal length of the probe is not a terminal portion or terminallength of the probe.

With respect to “distal,” a “distal portion” or a “distal end portion”of, for example, a probe disclosed herein includes a portion of theprobe intended to be near or in a patient when the probe is used on thepatient. Likewise, a “distal length” of, for example, the probe includesa length of the probe intended to be near or in the patient when theprobe is used on the patient. A “distal end” of, for example, the probeincludes an end of the probe intended to be near or in the patient whenthe probe is used on the patient. The distal portion, the distal endportion, or the distal length of the probe can include the distal end ofthe probe; however, the distal portion, the distal end portion, or thedistal length of the probe need not include the distal end of the probe.That is, unless context suggests otherwise, the distal portion, thedistal end portion, or the distal length of the probe is not a terminalportion or terminal length of the probe.

The term “logic” may be representative of hardware, firmware or softwarethat is configured to perform one or more functions. As hardware, theterm logic may refer to or include circuitry having data processingand/or storage functionality. Examples of such circuitry may include,but are not limited or restricted to a hardware processor (e.g.,microprocessor, one or more processor cores, a digital signal processor,a programmable gate array, a microcontroller, an application specificintegrated circuit “ASIC”, etc.), a semiconductor memory, orcombinatorial elements.

Additionally, or in the alternative, the term logic may refer to orinclude software such as one or more processes, one or more instances,Application Programming Interface(s) (API), subroutine(s), function(s),applet(s), servlet(s), routine(s), source code, object code, sharedlibrary/dynamic link library (dll), or even one or more instructions.This software may be stored in any type of a suitable non-transitorystorage medium, or transitory storage medium (e.g., electrical, optical,acoustical or other form of propagated signals such as carrier waves,infrared signals, or digital signals). Examples of a non-transitorystorage medium may include, but are not limited or restricted to aprogrammable circuit; non-persistent storage such as volatile memory(e.g., any type of random access memory “RAM”); or persistent storagesuch as non-volatile memory (e.g., read-only memory “ROM”, power-backedRAM, flash memory, phase-change memory, etc.), a solid-state drive, harddisk drive, an optical disc drive, or a portable memory device. Asfirmware, the logic may be stored in persistent storage.

Referring to FIG. 1, a block diagram depicting various elements of amagnetic-based tracking system 1110 for tracking of an ultrasound probe1140 and generation of a three-dimensional (3D) image is shown inaccordance with one example embodiment of the present invention. Asshown, the system 1110 generally includes a console 1120, an ultrasoundprobe 1140, and a reference-device 800, where each of the probe 1140 andthe reference device 800 are configured to be communicatively coupled tothe console 1120. The console 1120 is shown to include one or moreprocessors 1122, a communication interface 1142, a display 1130 andnon-transitory, computer-readable medium (“memory”) 1150. The memory1150 is configured to store logic modules including an ultrasound probedata receiving logic 1152 and a magnetometer data receiving logic 1156.Further, the memory 1150 may include data stores such as the ultrasoundprobe data 1154, magnetometer data 1158 and the relation data 1160.

In some embodiments, the processor 1122, including non-volatile memorysuch as EEPROM for instance, is included in the console 1120 forcontrolling system function during operation of the system 1110, thusacting as a control processor. The display 1130 in the presentembodiment may be integrated into the console 1120 and is used todisplay information to the clinician while using the ultrasound probe1140. In another embodiment, the display 1130 may be separate from theconsole 1120.

The ultrasound probe 1140 uses an ultrasound acoustic transducer oracoustic array to produce and receive echoes that can be converted intoan image. For example, in some embodiments, the ultrasound probe mayinclude an ultrasound generation device including an ultrasound acousticstack or other various modalities of ultrasound generation (e.g.,microelectromechanical systems (MEMS) based, etc.). Within themagnetic-based tracking system, in some embodiments, the ultrasoundprobe 1140 may additionally include a magnetometer for measuring amagnetic field strength. In some embodiments, the ultrasound probe 1140may include a reference magnet.

The reference device 800 is a device that, in some embodiments, containsthe reference magnet 700 of the magnetic-based tracking system 1110. Insome embodiments, the reference device 800 may be a cuff-like structurethat contains the reference magnet 700 and wraps around a body segment(e.g., bodily appendage, torso, mid-section, chest, etc.) to be imaged.In other embodiments, the reference device 800 may include a U-shapedstructure that contains the reference magnet 700 and is configured toallow the bodily appendage to be placed within the U-shaped structurefor imaging. In other embodiments, the reference device 800 may includea magnetometer.

FIG. 2A illustrates a top view of the magnetic-based tracking system1110 including the ultrasound probe 1140 of FIG. 1 according to someembodiments. The ultrasound probe 1140 is employed in connection withthe reference device 800 in order to track the positioning of the probe1140 as it is moved about a surface of a patient. As shown, the probe1140 is configured to be moved along a surface (i.e., skin) 1220 of abodily appendage 1222 of a patient. The reference device 800 may bedeployed within a threshold distance to the bodily appendage 1222. Insome embodiments, the reference device 800 may comprise a hard, plasticcasing configured in a “U-shape” such as shown in FIG. 1, where aninterior is formed by the convex portion of the configuration. In suchan instance, the bodily appendage 1222 (e.g., an arm) may be placedwithin the interior.

The magnetic-based tracking system 1110 can be used, in someembodiments, in preparation for insertion of the needle and/or catheterinto the vasculature. Specifically, the system 1110 employs thecombination of the probe 1140 and the reference device 800 to track thepositioning of the probe 1140 in relation to a reference magnet 700 ofthe reference device 800. By tracking the positioning of the probe 1140in relation to the reference magnet 700, the system 1110 is able torelate each ultrasound image obtained by the probe 1140 to a positioningof the probe 1140 on the bodily appendage 1222. By relating anultrasound image to a positioning on the appendage 1222, the system 1110may then stitch together the set of ultrasound images to form a 3Dvisualization of the bodily appendage 1222. Such a 3D visualizationgives real-time 3D ultrasound guidance and assists in reducingcomplications typically associated with such introduction, includinginadvertent arterial puncture, hematoma, pneumothorax, etc.

In some embodiments, the ultrasound probe 1140 includes a head 1180 thathouses an ultrasound acoustic transducer or acoustic array 1144 forproducing ultrasonic pulses and for receiving echoes thereof afterreflection by the patient's body when the head is placed against thepatient's skin 1220, wherein each echo may be converted into an image.The ultrasound acoustic transducer or acoustic array 1144 may be inoperable communication with the console 1120 for storing ultrasoundimages. In some embodiments, the ultrasound probe 1140 includes a sensorcomponent, such as the magnetometer 1192, for detecting the position ofthe reference magnet 700 during ultrasound imaging procedures, such asthose described above. As will be described in further detail below, themagnetometer 1192 may be embedded within the head 1180 of the ultrasoundprobe 1140. The magnetometer 1192 is configured to detect a magneticfield associated with the reference magnet 700 and enable the system1110 to track the reference magnet 700 as it relates to the magnetometer1192. In the present embodiment, the magnetometer 1192 is disposed in aplanar configuration in the head 1180 of the ultrasound probe 1140,though it is appreciated that the magnetometer 1192 can be disposed inother configurations within the ultrasound probe 1140. The magnetometer1192 may exist in a paired longitudinal configuration with theultrasound acoustic transducer or acoustic array 1144. In otherembodiments, the magnetometer 1192 may exist in a paired latitudinalconfiguration, paired circular configuration or a combination thereofwith the ultrasound acoustic transducer or acoustic array 1144.

In some embodiments, the magnetometer 1192 may include a series ofmagnetometers arranged in a configuration to track the reference magnet700. In one embodiment, the magnetometer 1192 includes three orthogonalsensor coils for enabling detection of a magnetic field in three spatialdimensions (not shown). An example of a 3D magnetic sensor is onemanufactured by Honeywell Sensing and Control of Morristown, N.J.Further, the magnetometer 1192 in one embodiment are configured asHall-effect sensors, though other types of magnetic sensors could beemployed.

The magnetometer 1192 may be in communication with the console 1120 forstoring information about the position of the magnetometer 1192 inrelation to the reference magnet 700, which will be described in moredetail herein.

In some embodiments, as illustrated in FIGS. 2A and 2B, the referencemagnet 700 generates a magnetic field that can be detected and measuredusing the magnetometer 1192. As the ultrasound probe 1140 is moved alongthe surface 1220 of the bodily appendage 1222, the strength of themagnetic field as detected by the magnetometer 1192 changes. Thedetected strength of the magnetic field is recorded along with anultrasound image captured by the probe 1140, where both are transmittedto the console 1120 for storage and processing, which may includegeneration of a 3D image comprised of a plurality of ultrasound images.In some embodiments, the reference magnet 700 includes an electromagnetthat is coupled to a power supply (not shown). In such an embodiment,the strength of the magnetic field can be increased which allows theultrasound probe 1140 to be used at greater distances from the referencemagnet 700.

FIG. 2A shows the bodily appendage 1222 that is cradled within theinterior of the reference device 800. It can be appreciated that thebodily appendage 1222 can be any bodily appendage and further understoodthat reference-devices having various sizes may be deployed with thesystem 1110 (i.e., to enable the system to be used with differentappendages). In some embodiments, reference devices may be specificallyconfigured for use with specific appendages (e.g., arms, legs) or otherbody portions (e.g., torso). In addition, some references devices may bespecifically configured for use with children while others areconfigured for use with adults, where adult-sized reference devicesgenerally have a larger size than those configured for children). Insome embodiments, the bodily appendage 1222 may include but is notlimited to a lower arm, an upper arm, a lower leg, an upper leg, or afoot.

In some embodiments, the reference device 800 may be configured to fitbody segments such as a torso, multiple legs, or multiple arms. In someembodiments, the reference device 800 may be reusable and may be made ofdurable material such as nylon, other polyesters or harden plastics. Insome embodiments as illustrated in FIG. 2A, the reference device 800 maybe configured to surround multiple sides of a bodily appendage, whereinbodily appendage 1222 may be inserted through the open side of thereference device 800 with the appendage 1222 unobstructed for imaging.

As will be described in more detail below, the inclusion of thereference magnet 700 and the magnetometer 1192 in the system 1110provides numerous advantages over current ultrasound systems,specifically with respect to generation of visualizations based on theobtained ultrasound data. Briefly, the reference magnet 700 generates amagnetic field that is detectable by the magnetometer 1192 in theultrasound probe 1140. As the reference magnet 700 remains stationaryrelative to the patient's appendage during use of the ultrasound probe1140, the reference magnet 700 acts as a reference point for theultrasound probe 1140. Thus, the ultrasound probe 1140 may bespecifically configured to associate a detected magnetic field strengthwith a particular ultrasound image (received as an echo). Further, theultrasound probe 1140 may be configured to continuously associate astrength of a detected magnetic field with an obtained ultrasound image.The associated pairings of {detected magnetic field strength, ultrasoundimage} may be provided to the console 1130 such that the logic of whichmay generate a 3D visualization by stitching the ultrasound imagestogether based on the magnetic field strength associated with each. Inother words, the logic of the console 1130 may properly align theultrasound images based on the detected magnetic field strengthassociated with each. In particular, the detected magnetic fieldstrength provides an indication of a location on the patient's appendagein relation to the stationary reference magnet 700, which is used toalign the ultrasound images.

In one particular embodiment, the reference device 800 may fullysurround the bodily appendage 1222, for example, a sleeve-like structurethat completely encloses the bodily appendage 1222 to be imaged. In suchan embodiment, the reference device 800 may be constructed of a cloth,neoprene or mesh material and may be affixed in position by sliding thereference device 800 over the appendage 1222 to be imaged. In someembodiments, the reference device 800 may be configured to surround andbe secured to the bodily appendage 1222 to be imaged. For example, thereference device 800 may wrap completely around the upper arm, proximalthe elbow and be secured to the upper arm by a first piece of temporaryadhesive such as double-sided tape or Velcro (see FIG. 3A).

In some embodiments, as illustrated in FIG. 2B, the reference device 800may be in communication with a remote-controlled tourniquet 870 tocompress the bodily appendage 1222, and a remote-controlled heatingelement 872 to engorge blood vessels during use.

Referring to FIG. 3A, a side view of the magnetic-based tracking system1110 including the ultrasound probe 1140 of FIG. 2 is shown according tosome embodiments. In some embodiments, as the ultrasound probe 1140 ismoving on the surface of the skin 1220, the ultrasound acoustictransducer or acoustic array 1144 is capturing images while themagnetometer 1192 measures the strength of a detected magnetic field asgenerated by the reference magnet 700 and relays the magnetic fieldstrength information back to the console 1120 along with the capturedultrasound image. In other words, as the ultrasound probe 1140 is movedalong a patient's bodily appendage, the ultrasound probe 1140 transmitsan ultrasonic pulse and receives an echo, which is translated intoinformation utilized to render a two-dimensional (2D) image of the echoreceived at a particular location on the bodily appendage. As disclosedherein, as the ultrasound probe 1140 transmits the ultrasonic pulse andreceives the echo, the magnetometer 1192 concurrently detects a magneticfield generated by the reference magnet 700 and records its strength,where concurrently means at least partially overlapping in time. Thus,the ultrasound probe 1140 captures information for creating a 2Dultrasound image at a particular location along the bodily appendage andrecords the strength of the magnetic field detected at that location.The ultrasound information and the magnetic field strength are coupledand transmitted to the console 1110 for storage and processing. Forinstance, a plurality of ultrasound information and magnetic fieldstrength couplings may be used to create a 3D visualization of at leasta portion of the bodily appendage by stitching 2D images created fromthe ultrasound information together based the magnetic field strengthdata. Specifically, the magnetic field strength data may be used toalign the 2D ultrasound images based on a reference point (e.g., thelocation of the reference magnet 700) through analysis of the magneticfield strength data.

As illustrated in FIG. 3A, the ultrasound probe 1140 takes a firstultrasound image 900 of the blood vessel 1226 using the ultrasoundacoustic transducer or acoustic array 1144 while the magnetometer 1192records a first strength 902 of the detected magnetic field generated bythe reference magnet 700. The ultrasound probe 1140 takes a secondultrasound image 904 of the blood vessel 1226 using the ultrasoundacoustic transducer or acoustic array 1144 while the magnetometer 1192records a second strength 906 of the detected magnetic field generatedby the reference magnet 700. Further, the ultrasound probe 1140 takes athird ultrasound image 908 of the blood vessel 1226 using the ultrasoundacoustic transducer or acoustic array 1144 while the magnetometer 1192records a third strength 910 of the detected magnetic field generated bythe reference magnet 700. The first image 900 and first strength 902,the second image 904 and second strength 906, and the third image 908and third strength 910 are each coupled and sent to the console 1120 forstorage and processing. Specifically, the console 1120 may create or addto a database of {ultrasound image, magnetic field strength} pairs. Thedatabase can then be used to construct a 3D image in a way that will bedescribed in more detail herein.

FIG. 3B provides an illustration of an alternative embodiment to thereference device 800 as illustrated in FIGS. 1-2. For example, in FIGS.1-2, the reference device 800 is illustrated as a cuff-like devicewherein a bodily appendage may be received at an interior of thereference device 800. In contrast, the reference device 300 of FIG. 3Bis illustrated as a wearable device, e.g., an arm band, in which thereference magnetic 700 is incorporated. It should be understood that thevarious embodiments of reference devices discussed in FIGS. 1-5 areinterchangeable in the functionality described, especially with respectto associating a captured ultrasound image with a detected magneticfield strength.

As mentioned above, the system 1110 in the present embodiment may beconfigured to detect the position and movement of the ultrasound probe1140. In particular, the magnetometer 1192 in the probe 1140 isconfigured to detect a magnetic field generated by or otherwiseassociated with the reference magnet 700. In some embodiments, when themagnetometer 1192 includes multiple magnetometers, each of themagnetometers may be configured to spatially detect the magnetic fieldin three-dimensional space. Thus, during operation of the system 1110,the magnetic field strength data of the reference magnet 700 sensed byeach of the magnetometers 1192 is forwarded to a processor, such as theprocessor 1122 of the console 1120 (FIG. 1), which, in conjunction withlogic, such as the magnetometer data receiving logic 1156 of the console1120 (FIG. 1) computes, in real-time, the position of the ultrasoundprobe 1140 in relation to the reference magnet 700 as well as thedistance between the reference magnet 700 and the ultrasound probe 1140.

FIG. 4 illustrates a side view of the magnetic based tracking system1110 including the ultrasound probe 1140 of FIG. 3A according to someembodiments. In various embodiments, the reference magnet 700 within thereference device 800 can be configured at different places in relationto the magnetometer 1192 and the ultrasound probe 1140. For example, ina first embodiment, the reference magnet 700 may be located in line withthe ultrasound probe 1140 or, in a second embodiment as is illustratedin FIG. 4, the reference magnet 700 may be located out of line with theultrasound probe 1140.

Referring to FIG. 5, a flowchart illustrating an exemplary method forcreating a 3D image using a magnetic-based tracking system of FIG. 1 isshown according to some embodiments. Each block illustrated in FIG. 5represents an operation performed in the method 500 of creating a 3Dimage using the magnetic-based tracking system of FIG. 1. Prior todetecting the proximity of the reference magnet to the magnetometer inthe probe, in one embodiment, is assumed that the magnetic-basedtracking system includes a probe having a body and a magnetometer, andthe reference magnet in a reference device that has been configured togenerate a magnetic field. It is further assumed that the magnetometeris configured to detect the magnetic field generated by the referencemagnet. Finally, it is assumed that the magnetic-based tracking systemincludes logic that is stored on non-transitory computer-readable mediumand, when executed by one or more processors, causes performance ofoperations associated with the proximity detection disclosed herein.

As an initial step in the method 500, the reference device is configuredaround the bodily appendage to be imaged (block 502). As is understood,the reference device may fully or partially surround the bodilyappendage to be imaged. In some embodiments, the reference device may befixed to the bodily appendage and may be placed above or below thebodily appendage for enhanced imaging.

The ultrasound probe is advanced on the skin surface of a patient toimage a target vessel (block 504). As is understood, the probe may bepositioned on the skin surface enabling the projection of an ultrasoundbeam toward the target vessel for imaging purposes (see FIG. 4). In someembodiments, the advancing step includes advancing the ultrasound probein any direction on the surface of the skin.

As the ultrasound probe is advanced toward the target vessel, theultrasound acoustic transducer or acoustic array within the head of theprobe captures images of the target vessel (block 506). In someembodiments, the capturing step includes capturing the ultrasound imagesto a database located on the console.

Simultaneously, the magnetometer of the probe detects a magnetic fieldstrength generated by or associated with the reference magnet that isincluded in the reference device (block 508). In some embodiments, whenthe magnetometer includes multiple magnetometers, the detecting stepincludes using all the magnetometers to detect the magnetic field.

The logic determines a distance between the reference magnet and themagnetometer of the probe based at least in part on the magnetic fieldstrength (block 510). As discussed above, the logic may be stored on aconsole, the probe or an alternative electronic device. In someembodiments, when the magnetometer includes multiple magnetometers, thelogic may determine the distance using the distance from eachmagnetometer.

Once the magnetic field strength between the reference magnet and themagnetometer of the probe is determined, the data is configured into adatabase that associates the distance between the reference magnet andthe magnetometer with the ultrasound image recorded at the exactdistance (block 512). In some embodiments, when the magnetometerincludes multiple magnetometers, the distance data from eachmagnetometer can be configured to be stored into the database.

Finally, in response to associating the magnetic field strength betweenthe reference magnet and magnetometer with the ultrasound image recordedat that exact distance, the logic stitches together multiple ultrasoundimages using the magnetic field strength associated with each specificimage to stitch the images together sequentially (block 514). Thestitching together of the multiple ultrasound images provides a 3D imageof the target.

Referring to FIG. 6, a block diagram depicting various elements of amagnetic-based tracking system 1112 for tracking of an ultrasound probe1146 and generation of a three-dimensional (3D) image is shown inaccordance with one example embodiment of the present invention. Asshown, the system 1112 generally includes a console 1120, an ultrasoundprobe 1146, and a reference-device 802, where each of the probe 1146 andthe reference device 802 are configured to be communicatively coupled tothe console 1120. The console 1120 is shown to include one or moreprocessors 1122, a communication interface 1142, a display 1130 andnon-transitory, computer-readable medium (“memory”) 1150. The memory1150 is configured to store logic modules including a magnetometer datareceiving logic 1156 and an ultrasound probe data receiving logic 1152.Further, the memory 1150 may include data stores such as themagnetometer data 1158, ultrasound probe data 1154, and the relationdata 1160. In some embodiments, the logic currently in the console 1120will perform the association of the time stamp from the ultrasoundimages data with the time stamps from the magnetic field strength data.The time stamps for the ultrasound images and the time stamps for themagnetic field strength data may be included in other embodimentsdiscussed herein, including for the purpose of confirming the ultrasoundimage and magnetic field strength association in the event of atransmission error from the ultrasound probe 1146 and reference device802 to the console 1120.

In some embodiments, the processor 1122, including non-volatile memorysuch as EEPROM for instance, is included in the console 1120 forcontrolling system function during operation of the system 1110, thusacting as a control processor. The display 1130 in the presentembodiment may be integrated into the console 1120 and is used todisplay information to the clinician while using the ultrasound probe1140. In another embodiment, the display 1130 may be separate from theconsole 1120.

In some embodiments, as illustrated in FIG. 6, the reference device 802includes the magnetometer 1194 and a power source 1162 to power themagnetometer 1194. In some embodiments, the ultrasound probe 1146includes the ultrasound acoustic transducer or acoustic array 1148 and areference magnet 702 that can be configured in the head 1182 of theultrasound probe 1146. The magnetometer 1194 as shown in FIG. 6 performsin a similar manner as discussed above with respect to magnetometer 1192of FIGS. 1-4. However, the magnetometer 1194 of FIG. 6 is incorporatedinto the reference device 802 where the reference device 802 having themagnetometer 1192 detects a reference magnet 702 strength data andtransmits such to the console 1120. The reference magnet 702 as shown inFIG. 6 performs in a similar manner as discussed above with respect toreference magnet 700 of FIGS. 1-4. However, the reference magnet 702 ofFIG. 6 is incorporated into the ultrasound probe 1146, where theultrasound probe 1146 does not collect the magnetic field strength data.

In some embodiments, the magnetic-based tracking system 1112 includesthe reference device 802 further including the magnetometer 1194 and theultrasound probe 1146 including the reference magnet 702 wherein thereference magnet 702 can be attached to the ultrasound probe 1146 via asleeve around a probe handle 1200 or otherwise attached. This embodimentallows the reference magnet 702 to be used with various ultrasoundprobes that do not include reference magnets configured inside the head1182 of the ultrasound probe 1146.

FIG. 7 illustrates a top view of the magnetic-based tracking system 1112including the ultrasound probe 1146 of FIG. 6 according to someembodiments. The ultrasound probe 1146 is employed in connection withthe reference device 802 in order to track the positioning of theultrasound probe 1146 as it is moved about a skin surface of a patient1220. As shown, the probe 1146 is configured to be moved along thesurface (i.e., skin) 1220 of a bodily appendage 1222 of a patient. Thereference device 802 may be deployed within a threshold distance to thebodily appendage 1222. In some embodiments, the reference device 802 maycomprise a hard, plastic casing configured in a “U-shape” such as shownin FIG. 6, where an interior is formed by the convex portion of theconfiguration. In such an instance, the bodily appendage 1222 (e.g., anarm) may be placed within the interior.

The magnetic-based tracking system 1112 can be used, in someembodiments, in preparation for insertion of the needle and/or catheterinto the vasculature. Specifically, the system 1112 employs thecombination of the probe 1146, the reference device 802 including themagnetometer 1194 to track the positioning of the ultrasound probe 1146in relation to a reference magnet 702 within the ultrasound probe 1146.By tracking the positioning of the probe 1146 in relation to thereference magnet 702, the system 1112 is able to relate each ultrasoundimage obtained by the probe 1146 to a positioning of the probe 1146 onthe bodily appendage 1222. By relating an ultrasound image to apositioning on the appendage 1222, the system 1112 may then stitchtogether the set of ultrasound images to form a 3D visualization of thebodily appendage 1222. Such a 3D visualization gives real-time 3Dultrasound guidance and assists in reducing complications typicallyassociated with such introduction, including inadvertent arterialpuncture, hematoma, pneumothorax, etc.

In some embodiments, the ultrasound probe 1146 includes a head 1182 thathouses an ultrasound acoustic transducer or acoustic array 1148 forproducing ultrasonic pulses and for receiving echoes thereof afterreflection by the patient's body when the head 1182 is placed againstthe patient's skin 1220, wherein each echo may be converted into animage. The ultrasound acoustic transducer or acoustic array 1148 may bein operable communication with the console 1120 for storing ultrasoundimages. In some embodiments, the ultrasound probe 1146 includes areference magnet 702 for providing the position of the ultrasound probe1146 during ultrasound imaging procedures, such as those describedabove. As will be described in further detail below, the referencemagnet 702 may be embedded within the head 1182 of the ultrasound probe1146. In the present embodiment, the reference magnet 702 is disposed ina planar configuration in the head 1182 of the ultrasound probe 1146,though it is appreciated that the reference magnet 702 can be disposedin other configurations within the ultrasound probe 1146. The referencemagnet 702 may exist in a paired longitudinal configuration, pairedlatitude configuration, paired circular configuration or a combinationthereof with the ultrasound acoustic transducer or acoustic array 1148.

In some embodiments, the magnetometer 1194 may include a series ofmagnetometers arranged in a configuration to track the reference magnet702. In one embodiment, the magnetometer 1192 includes three orthogonalsensor coils for enabling detection of a magnetic field in three spatialdimensions (not shown). An example of a 3D magnetic sensor is onemanufactured by Honeywell Sensing and Control of Morristown, N.J.Further, the magnetometer 1192 in one embodiment are configured asHall-effect sensors, though other types of magnetic sensors could beemployed.

The magnetometer 1194 may be in communication with the console 1120 forstoring information about the position of the reference magnet 702 inrelation to the magnetometer 1194, which will be described in moredetail herein.

The magnetometer 1194 is configured to detect a magnetic fieldassociated with the reference magnet 702 and enable the system 1112 totrack the reference magnet 702 as it relates to the magnetometer 1194.

Referring to FIG. 8, a side view of the magnetic-based tracking system1112 including the ultrasound probe 1146 of FIG. 7 is shown according tosome embodiments. In some embodiments, as the ultrasound probe 1146 ismoving on the surface of the skin 1220, the ultrasound acoustictransducer or acoustic array 1148 is capturing images while themagnetometer 1194 in the reference device 802 measures the strength of adetected magnetic field as generated by the reference magnet 702 in theultrasound probe 1146 and relays the magnetic field strength informationback to the console 1120 along with the captured ultrasound image.

In other words, as the ultrasound probe 1194 is moved along a patient'sbodily appendage, the ultrasound probe 1146 transmits an ultrasonicpulse and receives an echo, which is translated into information andtime stamped, utilized to render a two-dimensional (2D) image of theecho received at a particular location on the bodily appendage. Asdisclosed herein, as the ultrasound probe 1146 transmits the ultrasonicpulse and receives the echo, the magnetometer 1194 within the referencedevice 802 concurrently detects a magnetic field generated by thereference magnet 702 within the ultrasound probe 1146 and records itsstrength and time stamps the magnetic field strength, where concurrentlymeans at least partially overlapping in time. Thus, the ultrasound probe1146 captures information for creating a 2D ultrasound image at aparticular location along the bodily appendage and the magnetometer 1194records the strength of the magnetic field detected at that location.The ultrasound information including the time stamp and the magneticfield strength and time stamp are coupled and transmitted to the console1110 for storage and processing. For instance, a plurality of ultrasoundinformation, magnetic field strength and time stamp couplings may beused to create a 3D visualization of at least a portion of the bodilyappendage by stitching 2D images created from the ultrasound informationtogether based the magnetic field strength data and time stamping.Specifically, the magnetic field strength data and concurrent time stampmay be used to align the 2D ultrasound images based on a reference point(e.g., the location of the reference magnet 702) through analysis of themagnetic field strength data.

As illustrated in FIG. 8, the ultrasound probe 1146 takes a firstultrasound image 912 of the blood vessel 1226 using the ultrasoundacoustic transducer or acoustic array 1148 that is time stamped whilethe magnetometer 1194 in the reference device 802 records a firststrength 914 that is time stamped of the detected magnetic fieldgenerated by the reference magnet 702 within the ultrasound probe 1146.The ultrasound probe 1146 takes a second ultrasound image 916 of theblood vessel 1226 using the ultrasound acoustic transducer or acousticarray 1148 that is time stamped while the magnetometer 1194 records asecond strength 918 that is time stamped of the detected magnetic fieldgenerated by the reference magnet 702. Further, the ultrasound probe1146 takes a third ultrasound image 920 of the blood vessel 1226 usingthe ultrasound acoustic transducer or acoustic array 1148 that is timestamped while the magnetometer 1194 records a third strength 922 that istime stamped of the detected magnetic field generated by the referencemagnet 702. The first image 912 and first strength 914, the second image916 and second strength 918, and the third image 920 and third strength922 are each coupled and sent to the console 1120 for storage andprocessing. Specifically, the console 1120 may create or add to adatabase of {(ultrasound image, time stamp) (magnetic field strength,time stamp)} pairs. The database can then be used to construct a 3Dimage in a way that will be described in more detail herein.

FIG. 9 illustrates a side view of the magnetic based tracking system1112 including the ultrasound probe 1146 and reference device 802 ofFIG. 8 according to some embodiments. The magnetometer 1194 within thereference device 802 can be configured at multiple places in relation tothe reference magnet 702 and the ultrasound probe 1146. For example, themagnetometer 1194 may be located in line with the ultrasound probe 1146and reference magnet 702 or the magnetometer 1194 may be located out ofline with the ultrasound probe 1146 and reference magnet 702 asillustrated in FIG. 4.

Referring to FIG. 10, a flowchart illustrating an exemplary method 600for creating a 3D image using a magnetic-based tracking system 1112 ofFIG. 6 is shown according to some embodiments. Each block illustrated inFIG. 10 represents an operation performed in the method 1000 of creatinga 3D image using the magnetic-based tracking system of FIG. 6. Prior todetecting the proximity of the reference magnet in the ultrasound probeto the magnetometer in the reference device, in one embodiment, isassumed that the magnetic-based tracking system includes a probe havinga body and a reference magnet configured to generate a magnetic field,and the magnetometer in a reference device that has been configured todetect the magnetic field generated by the reference magnet. Finally, itis assumed that the magnetic-based tracking system includes logic thatis stored on non-transitory computer-readable medium and, when executedby one or more processors, causes performance of operations associatedwith the proximity detection disclosed herein.

As an initial step in the method 600, the reference device is configuredaround the bodily appendage to be imaged (block 602). As is understood,the reference device may fully or partially surround the bodilyappendage to be imaged. In some embodiments, the reference device may befixed to the bodily appendage and may be placed above or below thebodily appendage for enhanced imaging.

The ultrasound probe is advanced on the skin surface of a patient toimage a target vessel 1226 (block 604). As is understood, the probe maybe positioned on the skin surface enabling the projection of anultrasound beam toward the target vessel for imaging purposes (see FIG.9). In some embodiments, the advancing step includes advancing theultrasound probe in any direction on the surface of the skin.

As the ultrasound probe is advanced toward the target vessel, theultrasound acoustic transducer or acoustic array within the head of theprobe captures images of the target vessel (block 606). In someembodiments, the capturing step includes capturing the ultrasound imagesto a database located on the console.

Simultaneously, the magnetometer of the reference device detects amagnetic field strength generated by or associated with the referencemagnet that is included in or on the ultrasound probe (block 608). Insome embodiments, when the magnetometer includes multiple magnetometers,the detecting step includes using all the magnetometers to detect themagnetic field.

The logic determines a distance between the reference magnet of theprobe and the magnetometer of the reference device based at least inpart on the magnetic field strength (block 610). As discussed above, thelogic may be stored on a console, the probe or an alternative electronicdevice. In some embodiments, when the magnetometer includes multiplemagnetometers, the logic may determine the distance using the distancefrom each magnetometer.

Once the magnetic field strength between the reference magnet of theprobe and the magnetometer of the reference device is determined, thedata is configured into a database that associates the distance betweenthe reference magnet and the magnetometer with the ultrasound imagerecorded at the exact distance and the time stamp of the ultrasoundimage and the time stamp of the magnetic field strength (block 612). Insome embodiments, when the magnetometer includes multiple magnetometers,the distance data from each magnetometer can be configured to be storedinto the database.

Finally, in response to associating the magnetic field strength betweenthe reference magnet and magnetometer with the ultrasound image recordedat that exact distance, the logic stitches together multiple ultrasoundimages using the magnetic field strength associated with each specificimage and the time stamps to stitch the images together sequentially(block 614). The stitching together of the multiple ultrasound imagesprovides a 3D image of the target.

In addition to the above described embodiments, the inventive conceptsmay also be utilized in further embodiments such as, but not limited to,those described below. For example, in one embodiment, the trackingstructure of the magnetometer, the reference magnet and ultrasound probeare used to provide 31) views/guidance of anatomical structures intendedfor access with a needle (e.g. nerve blocks, drainage sites, biopsysites, etc.).

In another embodiment, the combination the tracking structure and aneedle tracking system is used to capture the trajectory of the needlein 3D. In furtherance of such an embodiment, the system 1110 may recordof the path of the needle to the target structure (e.g. center ofvessel, nerve bundle, drainage site, pneumothorax, etc.). In yet anotherembodiment, a magnetic or electromagnetic needle guidance system is usedto visually track within the 3D scan.

In another embodiment, the 3D scan may be viewed by a clinician througha virtual, augmented or mixed-reality system. In furtherance of such anembodiment, the clinician can evaluate or track needles, wires or toolsin real time.

In some embodiments, the tracking structure could be combined withvessel ID methods and Doppler capabilities to map veins and arteriesobserved during the scanning process.

In another embodiment, the tracking structure could be combined with ahands-free ultrasound probe to enable a live image at the desiredinsertion location and a 3D pre-scanned image. The hands-free probe withthe combination of the tracking structure would allow the clinician touse both hands for procedural device manipulation of the needle andpatient interaction of skin stabilization or limb extensionsimultaneously.

In some embodiments, the remote-controlled tourniquet may be a separatecuff from the reference device 800. In some embodiments, theremote-controlled heating element may be a separate cuff from thereference device 800.

In another embodiment, the tracking structure, in conjunction with theultrasound, could be used to create a baseline image of the lungs and apost-procedure scan to identify changes in the lungs (e.g. sliding lungbehavior associated with pneumothorax or improved blood flow). Infurtherance of such an embodiment, this combination could be combinedwith additional methods to identify or highlight structures of interestsuch as nerve bundles in the 3D scans.

In another embodiment, the tracking structure could include radio-opaquemarkers pursuant to the correlation to or image combination with X-ray,fluoroscopy or a combination thereof.

While some particular embodiments have been disclosed herein, and whilethe particular embodiments have been disclosed in some detail, it is notthe intention for the particular embodiments to limit the scope of theconcepts provided herein. Additional adaptations and/or modificationscan appear to those of ordinary skill in the art, and, in broaderaspects, these adaptations and/or modifications are encompassed as well.Accordingly, departures may be made from the particular embodimentsdisclosed herein without departing from the scope of the conceptsprovided herein.

1. A magnetic-based tracking system for tracking an ultrasound probe tocreate a three-dimensional (3D) visualization, the system comprising: areference device including a reference magnet; an ultrasound probecomprising a magnetometer configured to detect a magnetic fieldgenerated by the reference magnet, wherein the ultrasound probe isconfigured to couple a first ultrasound image with a first magneticfield strength, and wherein: the first ultrasound image is received at afirst time, and the first magnetic field strength is detected at thefirst time; and a console including a processor and non-transitorycomputer-readable medium having stored thereon a plurality of logicmodules that, when executed by the processor, are configured to performoperations including: receiving a plurality of couplings of ultrasoundimages and detected magnetic field strengths; recording the plurality ofcouplings of ultrasound images and detected magnetic field strengths;and generating the 3D visualization from the ultrasound images byaligning each of the ultrasound images in accordance with acorresponding detected magnetic field strength.
 2. The magnetic basedtracking system according to claim 1, wherein the reference device is acuff like structure that wraps around a body segment to be imaged. 3.The magnetic based tracking system according to claim 1, wherein thereference device is a U-shaped structure and is configured to allow thebody segment to be placed within the U-shaped structure for imaging. 4.The magnetic based tracking system according to claim 1, furthercomprising a needle tracking system configured to capture a trajectoryof a needle in three-dimensions, wherein the needle tracking system isconfigured to record of a path of the needle to a target structure. 5.The magnetic based tracking system according to claim 4, wherein thetarget structure is one of a vessel center, a nerve bundle, a drainagesite, or a pneumothorax.
 6. The magnetic based tracking system accordingto claim 1, further comprising a magnetic or electromagnetic needleguidance system configured to track a path of a needle to a targetstructure.
 7. The magnetic based tracking system according to claim 1,wherein the 3D visualization is rendered via a virtual, augmented ormixed-reality system.
 8. The magnetic based tracking system according toclaim 1, further comprising: a vessel identification system; and aDoppler system, wherein the vessel identification system and the Dopplersystem are configured to map one or more veins or one or more arteries.9. The magnetic based tracking system according to claim 1, wherein theultrasound probe is a hands-free ultrasound probe.
 10. The magneticbased tracking system according to claim 1, further comprising aremote-controlled tourniquet configured to compress a portion of thebody segment being imaged by the ultrasound probe.
 11. The magneticbased tracking system according to claim 1, further comprising aremote-controlled heating element configured to engorge blood vessels.12. A magnetic based tracking system for tracking an ultrasound probe tocreate a three-dimensional (3D) visualization, the system comprising: areference device including a magnetometer that detects a magnetic fieldgenerated by a reference magnet and creates a timestamp for a magneticfield strength reading; an ultrasound probe including an ultrasoundacoustic transducer or acoustic array that acquires ultrasound imagesand creates a timestamp for each specific ultrasound image, and areference magnet configured to generate a magnetic field; and a consoleincluding a processor and non-transitory computer-readable medium havingstored thereon a plurality of logic modules that, when executed by theprocessor, are configured to perform operations including: receiving aplurality of ultrasound images and detected magnetic field strengths,coupling a plurality of ultrasound images and detected magnetic fieldstrengths by their timestamps, recording the plurality of couplings ofultrasound images and detected magnetic field strengths, and generatingthe 3D visualization from the ultrasound images by aligning each of theultrasound images in accordance with a corresponding detected magneticfield strength.
 13. The magnetic based tracking system according toclaim 12, wherein the reference device is a cuff like structure thatwraps around a body segment to be imaged.
 14. The magnetic basedtracking system according to claim 12, wherein the reference device is aU-shaped structure and is configured to allow the body segment to beplaced within the U-shaped structure for imaging.
 15. The magnetic basedtracking system according to claim 12, further comprising a needletracking system configured to capture a trajectory of a needle inthree-dimensions, wherein the needle tracking system is configured torecord of a path of the needle to a target structure.
 16. The magneticbased tracking system according to claim 15, wherein the targetstructure is one of a vessel center, a nerve bundle, a drainage site, ora pneumothorax.
 17. The magnetic based tracking system according toclaim 12, further comprising a magnetic or electromagnetic needleguidance system configured to track a path of a needle to a targetstructure.
 18. The magnetic based tracking system according to claim 12,wherein the 3D visualization is rendered via a virtual, augmented, ormixed-reality system.
 19. The magnetic based tracking system accordingto claim 12, further comprising: a vessel identification system; and aDoppler system, wherein the vessel identification system and the Dopplersystem are configured to map one or more veins or one or more arteries.20. The magnetic based tracking system according to claim 12, whereinthe ultrasound probe is a hands-free ultrasound probe.
 21. The magneticbased tracking system according to claim 12, further comprising aremote-controlled tourniquet configured to compress a portion of thebody segment being imaged by the ultrasound probe.
 22. The magneticbased tracking system according to claim 12, further comprising aremote-controlled heating element configured to engorge blood vessels.23-37. (canceled)